Skip to content

Delete rows specified by CSV #840

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 8 commits into from
Feb 18, 2022
Merged

Delete rows specified by CSV #840

merged 8 commits into from
Feb 18, 2022

Conversation

krivard
Copy link
Contributor

@krivard krivard commented Feb 9, 2022

partially addresses remaining backend work for cmu-delphi/covidcast-indicators#1452

Prerequisites:

  • Unless it is a documentation hotfix it should be merged against the dev branch
  • Branch is up-to-date with the branch to be merged with, i.e. dev
  • Build is successful
  • Code is cleaned up and formatted

Summary

Add a function to database.py which, given a data source of covidcast rows to be deleted will:

  • figure out what covidcast id belongs to each row
  • figure out whether the row to be deleted is flagged for is_latest_issue
  • delete the corresponding row from covidcast
  • set the is_latest_issue flag on the next-most-latest issue for any latest entries deleted

Includes tests using both CSVs and tuples as inputs; for CSVs you need to have FILE privileges granted to the database user. I have the Dockerfile config change stashed in another branch for this, but I'm not sure whether we want to apply it in prod, so I'm having CI skip the associated test for now.

@krivard krivard force-pushed the krivard/delete_csvs branch from 23bcce1 to 6dc49bf Compare February 10, 2022 15:43
@krivard krivard requested a review from chinandrew February 10, 2022 16:05
@krivard krivard marked this pull request as ready for review February 10, 2022 16:05
krivard and others added 2 commits February 16, 2022 13:17
Copy link
Contributor

@chinandrew chinandrew left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:shipit: :shipit: :shipit:

@krivard krivard merged commit 4eee3ba into dev Feb 18, 2022
@krivard krivard deleted the krivard/delete_csvs branch February 18, 2022 17:49
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants